Image sensing apparatus and image processing method

ABSTRACT

An image sensing apparatus includes: an image sensing section for sensing an image of a subject; a detector for detecting a luminance of the subject; a compressor for compressing a dynamic range of the subject image; and a controller for controlling a compression characteristic to be used in compressing the dynamic range based on a detection result of the detector.

This application is a divisional application of application Ser. No.11/699,590, filed Jan. 29, 2007, which is based on Japanese PatentApplication No. 2006-22271 and No. 2006-49518 filed on Jan. 31, 2006 andFeb. 27, 2006, respectively, the contents of which are herebyincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image sensing apparatus, an imagesensing system, and an image sensing method, and more particularly to animage sensing apparatus, an image sensing system, and an image sensingmethod for changing a dynamic range compression characteristic inaccordance with a luminance of a subject.

2. Description of the Related Art

The pixel size of an image sensor is being decreased accompanied by arecent demand for a miniaturized and high pixel density image sensor. Asa result, the illuminance range (so-called dynamic range) of the imagesensor tends to be narrow, which may adversely affect the image quality.In view of this, there is a demand for securing a wide dynamic range ofthe image sensor.

In an attempt to solve the above drawback, various techniques areproposed. There is disclosed an image sensing apparatus (see e.g.Japanese Examined Patent Publication No. Hei 7-97841) for generating acomposite image with a wider dynamic range than the dynamic ranges ofindividual screen images by sensing the screen images with differentexposure amounts a certain number of times, selecting image areas havinga proper exposure level from these screen images for image synthesis.There is also disclosed an image sensor (e.g. Japanese Unexamined PatentPublication No. 2000-165755) provided with a charge voltage converter,with plural capacitances having different voltage dependences, forconverting a signal charge transferred from a photoelectric converter ofthe image sensor into a signal voltage to make the dynamic rangevariable. There is also disclosed a method (see e.g. Japanese UnexaminedPatent Publication No. 2000-165754) for extending the dynamic range,with use of an image sensor provided with a number of capacitances forholding signal charges of photodiodes, by reading out signal chargesacquired by one-time exposure a certain number of times while switchingover the capacitance value, and by implementing summation with respectto the readout signals.

There is also proposed a logarithmic conversion type image sensor(hereinafter, called as “linear-logarithmic sensor”, see e.g. JapaneseUnexamined Patent Publication No. 2002-77733) for converting aphotocurrent into a logarithmically compressed voltage, using asub-threshold characteristic of an MOSFET, wherein outputcharacteristics inherent to a solid-state image sensor i.e. a linearconversion operation of linearly converting an electric signalcommensurate with an incident light amount for output, and a logarithmicconversion operation of logarithmically converting the electric signalcommensurate with the incident light amount for output are automaticallyswitched over by supplying a specific reset voltage to the MOSFET.

Whereas a sensing device such as the linear-logarithmic sensor hassucceeded in securing a wide dynamic range, in the current technology, adisplay device such as a monitor has not succeeded in securing a widedynamic range, as compared with the sensing device. Even if a widedynamic range is obtained for an input image, the effect of the widedynamic range cannot be satisfactorily exhibited on the display device.In other words, the technique of securing the dynamic range in thedevice for displaying, transferring, or storing an image sensed by thesensing device is limited. Even if a wide dynamic range image isobtained by the various methods as proposed above, it is difficult toprocess the entirety of the image information obtained by these methods.Accordingly, there is a need for a process of adjusting the dynamicrange of an input image for the dynamic range of the device fortransferring, storing, or displaying the image, while retaining usefulinformation, relating to the wide dynamic range input image, which hasbeen acquired by the aforementioned methods, for instance, a process(hereinafter, called as “dynamic range compression process”) forcompressing the dynamic range of the input image in such a manner thatthe wide dynamic range input image can be properly displayed with thedynamic range for the display device.

As an example of the dynamic range compression process, there is known amethod for creating plural blurred images based on an original imageacquired by an image sensor, and creating a composite blurred imagebased on the obtained blurred images to compress the dynamic range ofthe original image based on the composite blurred image. For instance,Japanese Unexamined Patent Publication No. 2001-298619 discloses anexample of the dynamic range compression approach of varying aprocessing parameter for creating plural blurred images, or a processingparameter for creating a composite blurred image in compressing thedynamic range in accordance with a scene discrimination result regardingthe original image, or photographic information attached to the originalimage.

The above technology discloses the dynamic range compression method, butdoes not disclose a gradation conversion method i.e. a contrastcorrection method, which is a process to be executed independently ofthe dynamic range compression e.g. a process to be executed after thedynamic range compression. Generally, a conversion process using agradation conversion characteristic i.e. gamma (γ) is performed as thegradation conversion. Changing the gamma in such a manner as to increasethe contrast in a low luminance area so as to optimize the contrast in amain subject area may unduly narrow the gradation in a high luminancearea, which may lower the contrast in the high luminance area. Thedynamic range compression process including the aforementioned techniquehas a drawback that a high frequency component i.e. an edge portion isrelatively emphasized, because an illumination component primarilyincluding a low frequency component is compressed, thereby generating anunnatural image with a high sharpness i.e. an image with an awkwardresolution.

SUMMARY OF THE INVENTION

In view of the above problems residing in the conventional examples, itis an object of the invention to provide an image sensing apparatus, animage sensing system, and an image sensing method that enable to obtaina proper image despite a dynamic range compression.

An aspect of the invention is directed to an image sensing apparatusincluding: an image sensing section for sensing an image of a subject; adetector for detecting a luminance of the subject; a compressor forcompressing a dynamic range of the subject image; and a controller forcontrolling a compression characteristic to be used in compressing thedynamic range based on a detection result of the detector.

These and other objects, features and advantages of the presentinvention will become more apparent upon reading the following detaileddescription along with the accompanying drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is front view showing an external appearance of a digitalcamera, as an example of an image sensing apparatus embodying theinvention.

FIG. 1B is a rear view showing an external appearance of the digitalcamera as an example of the image sensing apparatus embodying theinvention.

FIG. 2 is a functional block diagram showing an example of an electricalconfiguration of the digital camera shown in FIGS. 1A and 1B.

FIG. 3 is a diagram showing a relation among an image sensing area, aphotometric area, and an AF area.

FIG. 4 is a circuit block diagram showing an image processor and itsperiphery in the first embodiment of the invention.

FIG. 5 is a flowchart showing an example of an operation of a dynamicrange compression process.

FIG. 6A is a graph for describing an approach of obtaining a compressioncharacteristic of an illumination component.

FIG. 6B is a graph for describing an approach of obtaining a compressioncharacteristic of an illumination component.

FIG. 7 is a flowchart showing an example of a method for determining acompression ratio in a second embodiment of the invention.

FIG. 8 is a graph showing a relation between an illumination componentand an illumination component after compression in the secondembodiment.

FIG. 9A is a graph for describing a method for obtaining plural imagesbased on a single image obtained by a sensing operation of alinear-logarithmic sensor in a third embodiment of the invention,specifically a graph showing a photoelectric conversion characteristicof the linear-logarithmic sensor.

FIG. 9B is a diagram showing a method for obtaining plural images basedon a single image obtained by a sensing operation of thelinear-logarithmic sensor in the third embodiment, specifically, a graphfor describing a method for creating plural images.

FIG. 10 is a schematic functional block diagram primarily concerning animage sensing process to be executed by a digital camera, as an exampleof an image sensing apparatus according to a fourth embodiment of theinvention.

FIG. 11 is a functional block diagram of a controller in the digitalcamera shown in FIG. 10.

FIG. 12 is a graph showing an example of a compression characteristic ofan illumination component to be used in a dodging process.

FIG. 13 is a graph showing an example of a gradation conversioncharacteristic to be used in a gradation conversion process.

FIG. 14 is a graph showing an example of the gradation conversioncharacteristic to be used in the gradation conversion process.

FIG. 15 is a flowchart showing an example of an operation concerning thegradation conversion process in accordance with a dodging parameter inthe fourth embodiment.

FIG. 16 is a functional block diagram of a controller in a digitalcamera according to a fifth embodiment of the invention.

FIG. 17 is a graph showing a relation between a filter size Sf and acorrection amount ΔSh.

FIG. 18 is a rear view showing a user interface of the digital camerashown in FIG. 10, specifically showing a state that sharpness isselected, and a pointer is moved to the level +1.

FIG. 19 is a flowchart showing an example of an operation concerning asharpness correction process in accordance with a dodging parameter inthe fifth embodiment.

FIG. 20 is a functional block diagram of an outline emphasis processorin a sharpness corrector to describe an example of a modification of thesharpness correction process in accordance with the dodging parameter inthe fifth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

In the following, preferred embodiments of the invention will bedescribed referring to the drawings.

First Embodiment

First, a digital camera as an example of an image sensing apparatusaccording to a first embodiment of the invention is described referringto FIGS. 1A, 1B, and 2. FIGS. 1A and 1B are external views of thedigital camera, wherein FIG. 1A is a front view, and FIG. 1B is a rearview. Referring to FIG. 1A, an exchange lens unit 20 is mounted on afront surface of a camera body 10 of the digital camera 1. A releasebutton 101 as an operation member for a sensing operation is provided onan upper surface of the camera body 10. A two-stage switch constitutedof an AF switch 101 a, which is actuated when the release button 101 ispressed down to a first predetermined position, and a release switch 101b, which is actuated when the release button 101 is pressed down to asecond predetermined position, is provided at an inner position of thecamera body 10 and below the release button 101. A flash section 102 isbuilt in an upper part of the camera body 10. A mode setting dial 112for setting the operation mode of the digital camera 1 is arranged onthe upper part of the camera body 10.

As shown in FIG. 1B, there are provided, on a rear surface of the camerabody 10, a power supply switch 111 for turning on or off the powersource of the digital camera 1, a condition changing dial 113 forchanging various setting conditions of the digital camera 1, a jog dial115 constituted of five switches provided at upper, lower, left, rightand middle positions to perform various settings at the operation modesof the digital camera 1, a viewfinder eyepiece lens element 121 a, and amonitor section 131 for displaying a recorded image or variousinformation relating to a photographing operation.

FIG. 2 is a functional block diagram showing an example of an electricalconfiguration of the digital camera 1 shown in FIGS. 1A and 1B. A maincontroller 150 for controlling the digital camera 1 controls overalloperations of the digital camera 1, and includes a CPU (CentralProcessing Unit) 151, a work memory 152, a storage 153, and a datamemory 154. The main controller 150 reads out various control programsfrom the storage 153 for outputting into the work memory 152 to controlthe respective parts of the digital camera 1 in accordance with thereadout program. The data memory 154 stores various data relating tophotometry or an AF (Auto-Focus) operation.

Also, the main controller 150 receives an input signal from theoperation members such as the power supply switch 111, the mode settingdial 112, the condition changing dial 113, the jog dial 115, the AFswitch 101 a, and the release switch 101 b. The main controller 150controls a photometric operation by communicating with a photometrymodule 122 provided on an optical viewfinder section 121, and controlsan AF operation by communicating with an AF module 144 in accordancewith the input signal. The main controller 150 controls a mirror driver143 to drive a reflex mirror 141 and a sub mirror 142, and controls ashutter driver 146 to drive a shutter 145, and controls the flashsection 102 in accordance with the input signal. Further, the maincontroller 150 controls an image sensing operation by communicating withan imaging controller 161, and is operative to display a sensed image orvarious information relating to a photographing operation on the monitorsection 131, and to display various information relating to thephotographing operation on an in-finder display section 132.

Also, the main controller 150 communicates, via an external interface(I/F) 185, data concerning the sensed image, control signals forcontrolling the digital camera 1, or the like with a personal computer(PC) or a personal digital assistant externally connected to the digitalcamera 1. Also, the main controller 150 controls overall operations ofthe exchange lens unit 20 by communicating with a lens controller 241for controlling focusing and zooming of a lens element 211, an aperturecontroller 222 for controlling an aperture of a diaphragm 221, and alens information storage 231 for storing information inherent to theexchange lens unit 20, by way of a lens interface 251 of the exchangelens unit 20, via a body-side body-lens communicator 172 provided on abody mount portion 171 of the camera body 10 for balance photometry, anda lens-side body-lens communicator 272 provided on a lens mount portion271 of the exchange lens unit 20, for communication between the camerabody 10 and the exchange lens unit 20.

An image sensor 162 is a sensing device for sensing a subject lightimage, and photoelectrically converts the subject light image formed onthe lens element 211 into image signals of color components of R, G, andB in accordance with an amount of the subject light image for outputtingthe image signals to an amplifier 163, which will be described later.The image signals are amplified by the amplifier 163, and then convertedinto digital data by a front end processor 164 (hereinafter, called as“FEP 164”), and are also subjected to a pre-processing such as blacklevel correction or fixed pattern noise (hereinafter, called as “FPN”)correction. Then, the processed data undergoes various image processingin an image processor 165 to be temporarily recorded into an imagememory 181, and finally recorded into a memory card 182. Theseoperations are controlled under the control of the main controller 150by the imaging controller 161. The imaging controller 161, the amplifier163, the FEP 164, and the image processor 165 constitute an imagingcircuit 160.

In this embodiment, there is used, as the image sensor 162, an imagesensor (hereinafter, also called as “linear-logarithmic sensor”according to needs) having a photoelectric conversion characteristicwith a linear conversion area where an output pixel signal i.e. anoutput electric signal generated by photoelectric conversion is linearlyconverted for output, when an incident luminance of the image sensor islow i.e. a dark image is sensed, and a logarithmic conversion area wherethe output pixel signal is logarithmically converted for output, whenthe incident luminance of the image sensor is high i.e. a bright imageis sensed, in other words, a linear photoelectric conversioncharacteristic in a low luminance area, and a logarithmic photoelectricconversion characteristic in a high luminance area. A sensing operationby the linear-logarithmic sensor enables to acquire an image with a widedynamic range. Acquiring an image with a wide dynamic range by the imagesensor 162 enables to easily obtain a compression characteristic asshown in FIG. 9A, which will be described later.

The linear-logarithmic sensor is, for instance, a so-called CMOS(complementary metal-oxide semiconductor) image sensor produced byadditionally providing a logarithmic conversion circuit with a P-type oran N-type MOSFET or a like device to a solid-state image sensorcomprised of a multitude of photoelectric conversion elements such asphotodiodes in a matrix so as to convert an output characteristic of thesolid-state image sensor in such a manner that the electric signal islogarithmically converted commensurate with the incident light amount,by utilizing a sub-threshold characteristic of the MOSFET. The imagesensor 162 may be a VMIS image sensor, a CCD (charge-coupled device)image sensor, or an equivalent sensor, in place of the CMOS imagesensor.

An image having a linear characteristic area and a logarithmiccharacteristic area obtained by the linear-logarithmic sensor is alsocalled as a “linear-logarithmic image” according to needs. A switchingpoint (also called as an “inflection point”) between the linearcharacteristic area and the logarithmic characteristic area concerningthe photoelectric conversion characteristic is arbitrarily controllableby a predetermined control signal to be inputted to the respective pixelcircuits of the image sensor 162. The image sensor 162 is not limited tothe linear-logarithmic sensor, but may be any image sensor including alinear sensor having plural linear characteristic areas with differentgradients, as far as a wide dynamic range image is obtainable. The widedynamic range image may be a wide dynamic range image created based onplural frames of images obtained by sensing an image, with use of ageneral well-known linear sensor, with different shutter speeds ordifferent aperture values, or may be a wide dynamic range image createdbased on a knee-processed image.

The image sensor 162 is not limited to the image sensor with the widedynamic range, but may be a general well-known image sensor having alinear photoelectric conversion characteristic such as a CCD imagesensor or a CMOS image sensor.

Next, an example of a photometric area to be measured by the photometrymodule 122, as an essential part of a detector for detecting theluminance of a subject is described referring to FIG. 3. FIG. 3 is adiagram showing a state that an image sensing area of the image sensor162, a photometric area of the photometry module 122, and an AF area ofthe AF module 144 are optically superimposed one over the other on thefield of a subject image.

The photometric area of the photometry module 122 includes fourteenphotometric sections i.e. a photometric section A0 corresponding to thesubstantially whole area within the image sensing area 162 z of theimage sensor 162 for photometry, and photometric sections A1 through A13obtained by dividing an inner area of the photometric section A0 intoplural sections (in this embodiment, thirteen sections) for photometry.The main controller 150 calculates, as multi-area photometric data,fourteen Bv values from Bv(0) to Bv(13), which are brightness valueswith respect to the respective photometric sections A0 through A13, orfourteen Ev values from Ev(0) to Ev(13), which are exposure values withrespect to the respective photometric sections A0 through A13, by APEX(Additive System of Photographic Exposure) calculation, based on outputsfrom respective areas of multi-area photometric devices mounted on thephotometry module 122. Detailed description on the photometriccalculation is omitted herein, since it is recited in e.g. JapaneseUnexamined Patent Publication No. 2005-192139.

The AF area includes three AF sections i.e. an AF section 144C formetering light on the central part of a field i.e. a subject, an AFsection 144L for metering light on a left portion of the subject, and anAF section 144R for metering light on a right portion of the subject.The main controller 150 determines e.g. a subject image within the AFarea at a nearest distance as a main subject based on AF data outputtedfrom the AF module 144 concerning these three AF sections, and controlsthe lens controller 241 to drive the lens element 211 for focusing onthe main subject to obtain a focusing signal.

The above describes the case that three AF sections and fourteenphotometric sections are provided to determine the position of the mainsubject by the AF operation. The configuration on the AF sections andthe photometric sections is not limited to the foregoing, and may bearbitrarily modified as far as such a modification does not depart fromthe gist of the embodiment. Also, the AF section to which the mainsubject is assumed to belong is not necessarily determined by the AFoperation, but may be determined by manual selection by the user i.e.the photographer, or other means.

FIG. 4 is a circuit block diagram showing the image processor 165 andits periphery in the first embodiment. Referring to FIG. 2, when the AFswitch 101 a is turned on i.e. depressed, the photometry module 122performs a multi-area photometry, and the AF module 144 performs an AFoperation. Then, when the release switch 101 b is turned on, the reflexmirror 141 and the sub mirror 142 are moved to their respectivemirror-up positions, and the shutter 145 is opened. Then, the subjectimage is formed on the image sensor 162 through the lens element 211.The multi-area photometry by the photometry module 122 functions as adetecting step in an image sensing method to be described later.

The foregoing describes the case that the photometry module 122 performsthe photometry. Alternatively, the photometry may be performed by aso-called video AE using an image sensed by the image sensor. In thealteration, for instance, an image is divided into plural blocksconsisting of m blocks in horizontal direction and n blocks in verticaldirection where m and n are positive integers, and an average luminanceIv(i,j) with respect to each block is obtained based on an average ofpixel values with respect to each block. The average luminance Iv(i,j)is expressed by: Iv(i,j)=log 2 (average pixel value with respect toblock (i,j)). Iv(i,j) may be used for calculation of a compressioncharacteristic in a similar manner as the Bv value or the Ev value.

Referring to FIG. 4, image data 162 k obtained by the image sensor 162is amplified by the amplifier 163, and then, the amplified image data issubjected to a pre-processing such as black level correction and FPNcorrection for conversion into digital image data 164 a. The digitalimage data 164 a is then outputted to the image processor 165. When thedigital image data 164 a is inputted to the image processor 165, thedigital image data 164 a undergoes white balance correction by a WB(white balance) corrector 501, color interpolation by a colorinterpolator 502, and color correction by a color corrector 503 to beoutputted to a dynamic range compressor 504, as corrected image data 503a. The respective processes to be executed by the WB corrector 501, thecolor interpolator 502, and the color corrector 503 may be generalwell-known processes.

Referring back to the operation of the main controller 150, the maincontroller 150 calculates a dynamic range compression parameter by thebelow-mentioned method based on the output from the photometry module122, and the calculated dynamic range compression parameter is outputtedto the dynamic range compressor 504, as a dynamic range compressioncharacteristic control signal 150 a. The corrected image data 503 awhich has been inputted to the dynamic range compressor 504 undergoesthe below-mentioned dynamic range compression process in accordance withthe dynamic range compression characteristic control signal 150 aoutputted from the main controller 150 for output as dynamic rangecompressed data 504 a. Thereafter, the dynamic range compressed data 504a undergoes gamma correction by a gamma corrector 505, color spacecorrection by a color space converter 506, and then compression with aproper compression ratio by a JPEG compressor 507 to be outputted to theimaging controller 161, as an image signal 165 a. The respectiveprocesses to be executed by the gamma corrector 505 and the color spaceconverter 506 may be general well-known processes. After having beenoutputted to the imaging controller 161, the image signal 165 a istemporarily recorded into the image memory 181 by the imaging controller161 under the control of the main controller 150, and then, is finallyrecorded into the memory card 182.

Next, the dynamic range compression process to be executed by thedynamic range compressor 504 is described referring to FIG. 5. FIG. 5 isa flowchart showing an example of an operation of the dynamic rangecompression process to be executed by the dynamic range compressor 504.First, in Step S101, an illumination component i.e. an illuminated imageor an illumination component image is extracted from the corrected imagedata 503 a. Generally, an image I is expressed as a product of anillumination component L and a reflectance component R i.e. areflectance image, according to the Retinex theory, as expressed by thefollowing formula (1).I=L*R  (1)where the symbol “*” denotes multiplication. The same definition isapplied to the below-mentioned description. Various approaches forextracting the illumination component L may be applied, including anapproach of extracting a low frequency component exclusively from theimage I using a low-pass filter, and an approach of extracting anillumination component, using a median filter, an ε filter, or the like(e.g. see “ε-separation non-linear digital filter and its application”by Harashima et al., Institute of Electronics, Information andCommunication Engineers (IEICE), Vol. J65-A, No. 4, pp, 297-304, Apr.1982).

Next, in Step S102, the reflectance component R is obtained by dividingthe image I by the illumination component L, as expressed by thefollowing formula (2).R=I/L  (2)where the symbol “/” denotes division. The same definition is applied tothe below-mentioned description.

In Step S103, a compression ratio c, as a compression parameter of theillumination component L to be used in the below-mentioned method, iscalculated based on the aforementioned multi-area photometric data. InStep S104, a compression process is performed with respect to theillumination component L. Specifically, assuming that the illuminationcomponent after compression is L′, the illumination component L′ isexpressed by the following formula (3).L′=n*L ^(C)  (3)where the symbol “n” denotes a regularization term, and the exponent “c”denotes the compression ratio c.

Then, in Step S105, by multiplying i.e. synthesizing the illuminationcomponent L′ after compression by the reflectance component R, asexpressed by the following formula (4), a dynamic range compressed imageI′ i.e. a synthesized image I′ is obtained.I=L′*R  (4)

In the above description, Step S103 functions as a calculating step inthe below-mentioned image sensing method, and Step S104 functions as acompressing step in the image sensing method.

Next, description is made regarding “calculation of compressionparameter of illumination component” to be executed in Step S103 in FIG.5, referring to FIGS. 6A and 6B. FIGS. 6A and 6B are graphs eachdescribing an approach for calculating the compression ratio c, as thecompression parameter of the illumination component L. Referring to FIG.6A, on the right portion of the graph, the axis of abscissas shows theimage data I obtained by the image sensor, and the axis of ordinateshows the illumination component L extracted from the image data I; andon the left portion of the graph, the axis of ordinate shows theillumination component L, and the axis of abscissas shows theillumination component L′ after compression. In this embodiment, theexposure control of the digital camera 1 is conducted in such a mannerthat the aperture value and the shutter speed are controlled so that theimage data corresponding to the area where the maximal luminance Bvmaxis given from the multi-area photometric data attains a maximal outputvalue of the image sensor 162. For sake of easy explanation, descriptionis made based on a premise that the illumination component L extractedfrom the image data I is proportional to the image data I.

First, description is made for a case that there is a large differenceD1 (=Bvmax1−Bvmin1) between the maximal subject luminance Bvmax1 and theminimal subject luminance Bvmin1 concerning the multi-area photometricdata, in other words, a case that a value obtained by subtracting theminimal value Bvmin1 from the maximal value Bvmax1 i.e. a differencevalue between the maximal value and the minimal value is large. Thedifference D1 may be an absolute value of the difference between themaximal value Bvmax1 and the minimal value Bvmin1. The same idea isapplied to the below-mentioned difference D2. When the aforementionedexposure control method is applied to the digital camera 1, the imagedata concerning the photometric section where the maximal value Bvmax1is given with respect to the multi-area photometry becomes a maximaloutput value to be outputted from the image sensor 162. Accordingly, theillumination component to be extracted from the image data becomes amaximal value L2 of the illumination component L i.e. an illuminationcomponent value L2. This results in setting the illumination componentto be extracted from the image data concerning the photometric sectionwhere the minimal value Bvmin1 is given with respect to the multi-areaphotometry to an extremely small value L11 i.e. an illuminationcomponent value L11, which results in formation of a dark distortedimage.

In the above condition, let us assume a compression characteristic curveCC1 capable of converting the illumination component value L11 into anillumination component value L1′ having proper brightness, andconverting the maximal value L2 of the illumination component into avalue L2′. Applying the compression characteristic curve CC1 enables toconvert the dark distorted image into an image with proper brightness,and to attain intended image reproducibility without likelihood that abrightest image portion may not be unnaturally bright. In the abovecondition, the compression characteristic curve CC1 that passes thepoint (L11, L1′), and the point (L2, L2′) in the left-side graphicalexpression of FIG. 6A is expressed by the following formula (5).L′=n1*L ^(C1)  (5)where the symbol “n1” denotes a regularization term, and the exponent“c1” denotes a compression ratio.

Similarly to the above, in the case where there is a small difference D2(=Bvmax2−Bvmin2) between the maximal subject luminance Bvmax2 and theminimal subject luminance Bvmin2 concerning the multi-area photometricdata, in other words, a case that a value obtained by subtracting theminimal value Bvmin2 from the maximal value Bvmax2 i.e. a differencevalue between the maximal value and the minimal value is small, theillumination component to be extracted from the image data concerningthe photometric section where the minimal value Bvmin2 is given withrespect to the multi-area photometry becomes an illumination componentvalue L21, which is larger than the minimal value Bvmin1. In this case,similarly to the above, assuming a compression characteristic curve CC2capable of converting the illumination component L21 into anillumination component value L1′ with proper brightness, and convertingthe maximal value L2 of the illumination component into a value L2′, thecompression characteristic curve CC2 is expressed by the followingformula (6).L′=n2*L ^(C2)  (6)where the symbol “n2” denotes a regularization term, and the exponent“c2” denotes a compression ratio. It should be noted that thecompression ratios c1 and c2 satisfy the relation: c1<c2.

The relationship between the illumination component L and theillumination component L′ after compression is shown in FIG. 6B. FIG. 6Bshows substantially the same graphical expression as the left-sidegraphical expression of FIG. 6A. Specifically, FIG. 6B shows thecompression characteristic curve CC1 in the case where the differencebetween the maximal value Bvmax and the minimal value Bvmin of thesubject luminance is large, and the compression characteristic curve CC2in the case where the difference between the maximal value Bvmax and theminimal value Bvmin of the subject luminance is small. As mentionedabove, within the coordinate space of FIG. 6B, the compressioncharacteristic of the illumination component is defined in such a mannerthat as the difference D between the maximal value and the minimal valueof the subject luminance is increased, the compression characteristiccurve with a large curvature i.e. the compression characteristic curveCC1 is applied, and that as the difference D is decreased, thecompression characteristic curve approximate to a linear curve i.e. thecompression characteristic curve CC2 is applied. The dynamic rangecompression characteristic is controlled based on the compressioncharacteristic curve.

Thus, calculating the compression ratio c, as the compression parameterof the illumination component L, using the difference D between themaximal value Bvmax and the minimal value Bvmin of the subject luminancebased on the multi-area photometric data enables to execute thecomputation of obtaining the compression ratio c at a very high speed,as compared with a conventional case that a bright or dark scene isdiscriminated based on the original image. This enables to perform thecompression process of the illumination component at a high speed, andto obtain a dynamic range compressed image having proper brightness.Also, obtaining a relation between the compression ratio c, and thedifference D between the maximal value Bvmax and the minimal value Bvminof the subject luminance in advance for storage in terms of a lookuptable (LUT) or a like format, and reading out the compression ratio cfrom the lookup table, according to needs, enables to simplify thecomputation of the compression ratio c and to expedite the computationprocessing.

Second Embodiment

In this section, a second embodiment of the invention is describedreferring to FIG. 7. FIG. 7 is a flowchart showing an example of amethod for determining the compression ratio c in the second embodiment.In this embodiment, the compression ratio c of the illuminationcomponent is determined by: defining a central part of an image screeni.e. a central area A7 of a photometric device 122 a shown in FIG. 3, asa main subject area; defining a part other than the central part as abackground area; and obtaining a main subject luminance Bvmain, and abackground luminance Bvsub e.g. an average luminance concerning thebackground area, based on multi-area photometric data. The main subjectarea is not limited to the central area A7, but may be an areadesignated by e.g. the user, or any area including a focusing areaobtained based on the result of an AF operation, as described referringto FIG. 3.

Referring to FIG. 7, first, in Step S201, the main subject luminanceBvmain is obtained based on the photometric data concerning the centralarea A7 obtained by multi-area photometry. Then, in Step S202, thebackground luminance Bvsub is obtained based on the average value of thephotometric data concerning the areas other than the central area A7obtained by the multi-area photometry. Then, in Step S203, it isconfirmed whether a difference ΔBv (=Bvmain−Bvsub) between the mainsubject luminance Bvmain and the background luminance Bvsub i.e. adifference value obtained by subtracting the background luminance Bvsubfrom the main subject luminance Bvmain is equal to or larger than 0, inother words, whether a scene is to be photographed, with the sun behindthe photographer, where the main subject luminance is equal to or largerthan the background luminance. If the scene is to be photographed withthe sun behind the photographer (YES in Step S203), in Step S204, thecompression ratio c of the illumination component is set to a smallvalue c1, and the routine is ended. By implementing the aboveoperations, the compression characteristic has a flat i.e. lineargradient with respect to the entirety of the compression characteristic,thereby securing proper gradations over the entirety of the image.

If the scene to be photographed is a backlit scene (NO in Step S203), inStep S205, it is confirmed whether a difference between the main subjectluminance Bvmain and the background luminance Bvsub: (Bvmain−Bvsub) isequal to or smaller than a predetermined threshold value e.g. a backlitdiscrimination value TH where TH is a negative number, in other words,whether the scene to be photographed is a severely backlit scene wherethe main subject luminance is considerably lower than the backgroundluminance. If it is judged that the scene to be photographed is theseverely backlit scene (YES in Step S205), in Step S206, the compressionratio c of the illumination component is set to a large value c3, andthe routine is ended. By implementing the above operations, the gradientof the compression characteristic in the low luminance area isincreased, and gradation performance i.e. contrast with respect to thesubject with a low luminance i.e. the low luminance area including themain subject image is secured. The backlit discrimination value TH maybe a predetermined fixed value, or may be arbitrarily set by the user.

If the scene to be photographed is neither a scene with the sun behindthe photographer nor a severely backlit scene, in other words, aslightly backlit scene where the main subject luminance is slightlylower than the background luminance, i.e., the main subject image isslightly dark (NO in Step S205), in Step S207, the compression ratio cof the illumination component is set to an intermediate value c2, andthe routine is ended. In this embodiment, the compression ratios c1, c2,and c3 satisfy the relation: c1<c2<c3. Thereby, in case of photographinga slightly backlit scene, an intermediate compression characteristicbetween the compression characteristic to be used in the scene with thesun behind the photographer, and the compression characteristic to beused in the severely backlit scene is adopted.

FIG. 8 shows a relationship between the illumination component L and theillumination component L′ after compression in the case where thecompression ratio of the illumination component defined by theabove-mentioned approach is used as a parameter. As described above, inthis embodiment, the dynamic range compression characteristic iscontrolled by: comparing the main subject luminance with the backgroundluminance; decreasing the compression ratio of the illuminationcomponent, as the main subject luminance is increased; and by increasingthe compression ratio, as the background luminance is increased, andmore particularly, by dividing the scene to be photographed intomultiple-stage scenes i.e. a scene with the sun behind the photographer,a severely backlit scene, and a slightly backlit scene, based on themain subject luminance and the background luminance.

In the above arrangement, as shown in FIG. 8, the compressioncharacteristic curve to be used in a scene with the sun behind thephotographer has a flat gradient over the entirety of the compressioncharacteristic, as shown by the curve CC1 expressed by: L′=n1*L^(C1),thereby securing gradation performance over the entirety of the image.The compression curve to be used in a severely backlit scene has a steepgradient in a low luminance area, as shown by the curve CC3 expressedby: L′=n3*L^(C3), thereby securing gradation performance with respect tothe subject with a low luminance, i.e. the low luminance area includingthe main subject image. The compression characteristic curve to be usedin a slightly backlit scene has an intermediate compressioncharacteristic between the compression characteristic to be used in thescene with the sun behind the photographer, and the compressioncharacteristic to be used in the severely backlit scene, as shown by thecurve CC2 expressed by: L′=n2*L^(C2), with an intermediate gradientbetween the gradient corresponding to the compression ratio c1 and thegradient corresponding to the compression ratio c3.

The above describes the case that the relationship between the mainsubject luminance and the background luminance concerning a subjectimage is defined in terms of three kinds of photographic scenes i.e. ascene with the sun behind the photographer, a slightly backlit scene,and a severely backlit scene. Alternatively, the relationship may bedefined more roughly e.g. in terms of two or less kinds of conditions,or more finely e.g. in terms of four or more kinds of conditions. Forinstance, the subject may be classified into a first main subject, asecond main subject, and a background.

Third Embodiment

In this section, a third embodiment of the invention is described. Inthis embodiment, dynamic range compression is performed by applying amethod called Goshtasby method. The Goshtasby method is, similarly tothe technology disclosed in Japanese Examined Patent Publication No. Hei7-97841, directed to sensing screen images with different exposureamounts a certain number of times to acquire a proper image, using theobtained screen images. The details of the Goshtasby method are recitedin e.g. “Fusion of Multi-Exposure Images”, by A. Goshtasby, Image andVision Computing, Vol. 23, 2005, pp. 611-618″.

In the third embodiment, a single image obtained by a linear-logarithmicsensor is used, in place of the plural images with different exposureamounts. In the following, the third embodiment is described referringto FIGS. 9A and 9B. FIGS. 9A and 9B are graphs for describing anapproach of obtaining plural images based on a single image obtained bythe linear-logarithmic sensor. Specifically, FIG. 9A is a graph showinga photoelectric conversion characteristic of the linear-logarithmicsensor, and FIG. 9B is a graph for describing an approach of creatingthe plural images. In these graphs, the axes of abscissas represent asubject luminance value, and the axes of ordinate represent a pixelvalue of the image sensor, wherein the axes of abscissa and the axes ofordinate both display the respective values in terms of a linearexpression.

Referring to FIG. 9A, the photoelectric conversion characteristic of thelinear-logarithmic sensor includes a linear characteristic i.e. acharacteristic 901 in FIG. 9A in a low luminance subject area, and alogarithmic characteristic i.e. a characteristic 902 in FIG. 9A in ahigh luminance subject area. Extending a portion corresponding to thelogarithmic characteristic 902 where the image is logarithmicallycompressed i.e. a logarithmic image to a linear characteristic 903 byimplementing a computation i.e. a characteristic conversion process withrespect to the linear-logarithmic image obtained by thelinear-logarithmic sensor enables to generate an image I with a linearcharacteristic over the entirety of the photoelectric conversioncharacteristic i.e. a totally linear image I. Also, photometric dataBv(1) through Bv(N) (in the example of FIG. 3, N=13) concerning therespective photometric sections obtained by multi-area photometry areobtained by implementing the method described referring to FIG. 3, or alike method.

Next, referring to FIG. 9B, images I1 through IN with their gradientsbeing corrected so that the photometric data Bv(1) through Bv(N) attaina proper pixel value (for instance, in the case where the maximal pixelvalue is 8-bit, the proper pixel value is 128) are created bycomputation based on the totally linear image I. For instance, in thecase where the photometric data concerning a certain photometric sectionhas a brightness value e.g. Bv1, an image I1 with a moderate gradient isobtained. The image I1 has a low contrast, although the original imageis satisfactorily reproduced without likelihood that an unnaturallybright image portion may not appear over the entirety of the image. Inthe case of the photometric data concerning a certain photometricsection has a brightness value e.g. Bv3, an image I3 with a steepgradient is obtained. The image I3 has a relatively high contrast in thelow luminance area, although all the bright image portions areunnaturally bright. In the case where the photometric data concerning acertain photometric section has an intermediate brightness value e.g.Bv2 between Bv1 and Bv3, an intermediate image I2 between the image I1and the image I3 is obtained.

Then, a dynamic range compression characteristic is determined, usingthe thus-obtained images I1 through IN. Since the operations to beimplemented thereafter are substantially the same as those recited inthe aforementioned technical document, the operations are brieflydescribed. First, each of the images I1 through IN is divided intoplural blocks i.e. block areas consisting of m blocks in horizontaldirection and n blocks in vertical directions where m and n are positiveintegers, and entropy E, which is a criterion representing the contrastof the image, is calculated with respect to each of the blocks. When theentropy of a color image at the block (j,k) is defined as Ec(jk), Ec(jk)is expressed by the following formula (7).

$\begin{matrix}{{{Ec}({jk})} = {{\sum\limits_{i = 0}^{255}{{- p_{i}^{r}}{\log\left( p_{i}^{r} \right)}}} + {\sum\limits_{i = 0}^{255}{{- p_{i}^{g}}{\log\left( p_{i}^{g} \right)}}} + {\sum\limits_{i = 0}^{255}{{- p_{i}^{b}}{\log\left( p_{i}^{b} \right)}}}}} & (7)\end{matrix}$where the symbol “p_(i)” denotes event probability of the pixel value iwithin the block, and the symbols “r”, “g”, and “b” respectively denotea red (R) pixel, a green (G) pixel, and a blue (B) pixel.

Next, a block where a maximal contrast is given i.e. a block image isselected from the blocks at positions corresponding to each other amongthe images I1 through IN, e.g. the blocks at the same positions amongthe images I1 through IN (in this embodiment, thirteen blocks) withrespect to each of the blocks. The image number I_(jk) attached to theimage where a maximal contrast is given is the number of the image wherea largest entropy is given among the N images concerning the block(i,j). If the thus selected m×n images are synthesized without anyprocessing, a resultant image may have incongruity i.e. boundariesbetween the block images where an unduly large difference concerning aluminance value remains. Accordingly, so-called blend processing isexecuted to generate an image with no or less incongruity. An outputimage O(x,y) obtained after the blend processing is expressed by thefollowing formula (8).

$\begin{matrix}{{O\left( {x,y} \right)} = {\sum\limits_{j = 1}^{m}{\sum\limits_{k = 1}^{n}{{W_{jk}\left( {x,y} \right)}{I_{jk}\left( {x,y} \right)}}}}} & (8)\end{matrix}$where the symbol “W_(jk)” denotes a weight expressed by the followingformula (9), and the symbols “I_(jk)(x,y)” denotes a pixel value of theimage where a maximal entropy is given concerning the block (j,k).

$\begin{matrix}{{W_{jk}\left( {x,y} \right)} = \frac{G_{jk}\left( {x,y} \right)}{\sum\limits_{j = 1}^{m}{\sum\limits_{k = 1}^{n}{G_{jk}\left( {x,y} \right)}}}} & (9)\end{matrix}$where the symbol “G_(jk)(x,y)” denotes a Gaussian expression representedby the following formula (10).

$\begin{matrix}{{G_{jk}\left( {x,y} \right)} = {\exp\left\{ {- \frac{\left( {x - x_{jk}} \right)^{2} + \left( {y - y_{jk}} \right)^{2}}{2\sigma^{2}}} \right\}}} & (10)\end{matrix}$where the symbol “x_(jk)” and “y_(jk)” i.e. “(x_(jk),y_(jk))” denotes acoordinate at the central pixel within the block, and the symbol “σ”denotes a standard deviation.

In this embodiment, the dynamic range compression characteristic iscontrolled by: creating plural images based on a detection result of thephotometric device with respect to a single image obtained by thelinear-logarithmic sensor; dividing each of the created images into anarbitrary number of blocks; and selecting an image with a high contrastwith respect to each of the blocks. Additionally implementing the blendprocessing enables to perform dynamic range compression desirably andreadily, thereby, generating a natural image. The series of operationsas mentioned above may be executed by the main controller 150, the imageprocessor 165, or an equivalent device. In other words, a proper imagecan be obtained despite the dynamic range compression. In the firstthrough the third embodiments, a parameter for dynamic range compressioncan be readily obtained by calculating the parameter for dynamic rangecompression based on information relating to the subject luminanceobtained in sensing the original image, and dynamic range compressioncan be performed without impairing the useful information included inthe original image.

Fourth Embodiment

FIG. 10 is a functional block diagram primarily showing an image sensingprocess to be executed by a digital camera 1 a, as an example of animage sensing apparatus according to a fourth embodiment of theinvention. As shown in FIG. 10, the digital camera 1 a includes an imageinput section 3, a main controller 4, a user I/F (interface) 5, and animage processor 6.

The image input section 2 is adapted to input an image into the digitalcamera 1 a, and includes a lens element 211, an image sensor 162, anamplifier 163, and an FEP 164 equivalent to the corresponding onesdescribed in the first through the third embodiments.

Similarly to the main controller 150, the main controller 4 includes anROM in which various control programs and the like are recorded, an RAMfor temporarily storing data, and a CPU for reading out the controlprograms and the like from the ROM for execution. The main controller 4controls overall operations of the digital camera 1 a. The maincontroller 4 calculates control parameters and the like required foroperating the respective parts of the digital camera 1 a including theimage sensor 162, based on various signals outputted from the respectiveparts of the digital camera 1 a, and controls the operations of therespective parts of the digital camera 1 a including a dodging processor61 to be described later, based on the control parameters. The maincontroller 4 is also connected to an image memory 181 or an equivalentdevice.

The user I/F 5 functions as an enter key through which the user isallowed to designate various operation commands, or is operative todisplay predetermined information. The user I/F 5 has a monitor section51 and an operation section 52. The monitor section 51 displays an imageinputted through the image input section 2, an image stored in the imagememory 181, predetermined operation information, setting information, orthe like. For instance, the monitor section 51 includes a liquid crystaldisplay (LCD) provided with a color liquid crystal display device. Theoperation section 52 is a member through which the user is allowed toenter an operation command to the digital camera 1 a. The operationsection 52 includes various switches i.e. buttons such as a releaseswitch, a photographing mode setting switch, and a menu selectionswitch. For instance, the image sensor 162 executes a series of imagesensing operations: sensing light from a subject in response to e.g.turning on of the release switch; performing a predetermined imageprocessing with respect to an image obtained by the sensing operation tooutput image data; and recording the image data into the image memory181. The monitor section 51 corresponds to the monitor section 131 inthe foregoing embodiments, and the operation section 52 corresponds tothe various members including the release switch 101, the AF switch 101a, the release switch 101 b, the power supply switch 111, the modesetting dial 112, the condition changing dial 113, and the jog dial 115in the foregoing embodiments.

As shown in e.g. FIG. 18, which is a rear view of the digital camera 1a, the user I/F 5 is configured in such a manner that the user isallowed to set i.e. enter various parameters (hereinafter, also calledas “image processing parameters”) relating to image processing such ascontrast correction amount, saturation correction amount, and sharpnesscorrection amount. Each of the image processing parameters has a scaleof adjustment in five stages e.g. from −2 to +2. As an actual inputoperation, for instance, let us presume that the user moves a selectedframe of image 511 to a predetermined position corresponding to anintended image processing parameter on the screen of the monitor section51 by manipulating a button or the like on the operation section 52, andthe image processing parameter to be corrected is selected i.e.activated. In the example of FIG. 18, sharpness is selected. Then, whenthe user shifts a pointer 512 from a default position (in this example,0) to an intended level position in the range from −2 corresponding to alow luminance priority position to +2 corresponding to a high luminancepriority position, the user is allowed to set a correction amount of theintended image processing parameter. In the example of FIG. 18, thesharpness correction amount is set to the level “+1”.

The image processor 6 is adapted to execute various image processingwith respect to the sensed image data inputted through the image inputsection 2 i.e. a digital image signal obtained by the FEP 164. The imageprocessor 6 includes the dodging processor 61, a gradation converter 62,and a sharpness corrector 63. The dodging processor 61 executes adodging process with respect to an image.

In this section, the idea on the dodging process is described. Asdescribed above, dynamic range compression has two meanings: one is tolocally adjust the contrast of an image by compressing an illuminationcomponent of the image; and the other is to literally compress thedynamic range while maintaining the contrast distribution with respectto the entirety of the image. In other words, the latter technique hasno relation to performing local adjustment on the contrast. The formertechnique is called as “dodging process”, and the latter technique iscalled as “dynamic range compression” to distinguish one from the other.Similarly to the process as mentioned above, the dodging processcomprises: extracting an illumination component from an original image;and performing a dynamic range compression with respect to theillumination component so that a new image, with its contrast beinglocally adjusted, is generated based on the illumination component i.e.a low frequency component and the reflectance component i.e. a highfrequency component after the dynamic range compression. The reflectancecomponent is extracted at the time when the illumination component isextracted. The dodging process to be executed in the fourth embodimentwill be described later in detail. The image processor in the fourthembodiment corresponds to the image processor 165 in the foregoingembodiments, and the dodging processor 61 is provided in place of thedynamic range compressor 504.

The gradation converter 62 is adapted to perform gradation conversion byperforming data conversion with respect to each of pixel data of R, G,and B of an input image i.e. a sensed image, using table information forgradation conversion, which is supplied from the main controller 4,specifically, a gradation conversion characteristic calculator 42, whichwill be described later. The gradation converter 62 in the fourthembodiment is provided in place of the gamma corrector 505 in theforegoing embodiments.

The sharpness corrector 63 is adapted to perform sharpness correctionwith respect to the input image by performing filter computation, usingpredetermined filter coefficient information for sharpness correction,which is supplied from the main controller 4. Similarly to the imageprocessor 165, the image processor 6 may include a color interpolator, acolor corrector, a WB corrector, an FPN corrector, or a black referencecorrector (all of which are not illustrated), in addition to theaforementioned functional parts.

In the fourth embodiment, the gradation conversion process is performedby determining a parameter (also, called as a “gradation conversionparameter”) relating to the gradation conversion process in accordancewith the parameters (also, called as “dodging parameters”) relating tothe dodging process. In other words, the fourth embodiment has a primaryfeature that the gradation conversion process is controlled inaccordance with the dodging parameters. In the following, the respectivefunction parts of the dodging processor 61 and the main controller 4 aredescribed.

The dodging processor 61 performs a dodging process with respect to animage inputted through the image input section 2 i.e. an image (also,called as a linear-logarithmic image, or a wide dynamic range image)obtained by a sensing operation of the image sensor 162. An originalimage I to which the dodging process is applied is also expressed by theaforementioned formula (1).

The dodging processor 61 extracts the illumination component L byapplying a so-called edge preserving filter such as a median filter oran ε filter, which is also called as a non-linear filter or anillumination extracting filter, with respect to the original image I.The dodging processor 61 also extracts the remainder component of theoriginal image I as the reflectance component R, which is obtained byextracting the illumination component L from the original image I. Theinformation relating to the filter size Sf of the edge preserving filteris supplied from the main controller 4. The filter size corresponds tothe image size constituted of plural pixels, which is determined basedon an idea as to for how many pixels an averaging process is to beexecuted in conducting a filter process with respect to the image. Forinstance, the filter size may correspond to a pixel group consisting of10 pixels, 20 pixels, etc.

In view of the above, it can be conceived that the dodging process is alocal computation process, in which computation is performed locallywith respect to each of image areas obtained by dividing an image intoplural areas. In this embodiment, the filter size of the edge preservingfilter is used as the size of each of the image areas i.e. an area size.Alternatively, the size of each block obtained by dividing an image intoplural blocks may be used. In this sense, information relating to thearea size including the filter size and the block size may be handled asthe dodging parameter. The local process may include e.g. a contrastcorrection process. In other words, the dodging process may be a processfor performing contrast correction with respect to each of the imageareas i.e. local contrast correction. In this sense, the area size maybe handled as a dodging parameter for the local contrast correction.

Next, the dodging processor 61 performs conversion according to thebelow-mentioned formula (11) i.e. dynamic range compression with respectto an area where the level of the extracted illumination component L isequal to or larger than a predetermined compression start level Ls(L≧Ls). Conversion according to the below-mentioned formula (12), whichis also included in the dynamic range compression, is performed withrespect to an area where the level of the extracted illuminationcomponent L is smaller than the compression start level Ls (L<Ls).L′=exp(log(L)*c)*n  (11)where the symbol “c” denotes a compression ratio, and the symbol “n”denotes a regularization term.L′=L  (12)

“L′” in the above formulae (11) and (12) are respectively indicated byillumination compression characteristics 310 and 320 in FIG. 12. Theillumination compression characteristic 310 is given as a curve thatpasses the point A (Ls, Os), which is indicated by the reference numeral311, where Ls is the predetermined compression start level of theillumination component L, and Os is a compression start level as anoutput value with respect to Ls, and that passes the point B (Lmax,Omax), which is indicated by the reference numeral 312, where Lmax is anupper limit value of the illumination component L, and Omax is an outputmaximal value with respect to the upper limit value Lmax. For instance,in the case where an output image obtained by the image sensor 162 is an8-bit image having gradations from 0 to 255, Omax is an output valuecorresponding to 255 in gradation level, i.e. a gradation value.Accordingly, the compression ratio c and the regularization term n inthe formula (11), which are the unknown numbers, are calculated by asimultaneous equation obtained by substituting the coordinate values atthe two points A and B, respectively. The information relating to Ls andLmax is given from the main controller 4.

After the aforementioned processes are executed, the dodging processor61 is operative to generate i.e. output an image I′, using thebelow-mentioned formula (13), which is equivalent to the formula (4),based on the illumination component L′ after the dynamic rangecompression, and the reflectance component R.I′=L′*R  (13)

The formula (13) may be conceived to be equivalent to the equation:I′=L′/L*I. However, the equation I′=I may be used, in place of theequation: I′=L′/L*I, concerning the area where the level of theillumination component L is smaller than the compression start level Ls(L<Ls).

The main controller 4 includes, as shown in FIG. 11, a dodging parametercalculator 41 and the gradation conversion characteristic calculator 42.The dodging parameter calculator 41 is adapted to calculate i.e. acquirethe dodging parameters such as Ls, Lmax, and the filter size Sf to beused in the dodging process. Ls is given as a value about twice as largeas a main subject luminance IC), which will be described later referringto FIG. 13, and is supposed to have proper brightness by performinggamma correction in the gradation converter 62 i.e. by conducting agradation conversion process using gamma. Ls may be a fixed value whichis predefined depending on the subject e.g. a person or a landscape, ormay be arbitrarily designated by the user by way of the user I/F 5 eachtime a sensing operation is conducted.

Lmax is given as a predetermined upper limit value for determining abrightness limit in the dodging process i.e. an upper limit of theluminance of the illumination component L to be compressed. Generally, aluminance maximal value concerning R, G, B components of the originalimage I i.e. a wide dynamic range image is used as Lmax. Alternatively,the maximal value of the illumination component L, Y % of the maximalvalue (considering a case that L max may be set to a value smaller thanthe maximal value of the illumination component to raise the contrast),X % of the maximal luminance value concerning R, G, B components, or thelike may be used. Lmax may be calculated each time a sensing operationis conducted, or may be a predetermined fixed value, or may bedesignated by the user by way of the user I/F 5 each time a sensingoperation is conducted. The filter size Sf may be given as apredetermined value, or a value dependent on the image, e.g. a valuedependent on Z % value of the image size or a space frequency of theimage. The dodging parameter calculator 41 sends the informationrelating to Ls, Lmax, and Sf to set the information in the dodgingprocessor 61. The information relating to the aforementioned values maybe stored in a parameter storage (not shown) provided e.g. in the maincontroller 4.

The gradation conversion characteristic calculator 42 calculates agradation conversion characteristic to be used in the gradationconversion process by the gradation converter 62, based on the dodgingparameters obtained by the dodging parameter calculator 41, and createsa conversion table e.g. a lookup table (LUT), in which the gradationconversion characteristic is described, i.e., the gradation conversionusing the gradation conversion characteristic is applied. The gradationconversion characteristic calculator 42 sends the information relatingto the created conversion table to the gradation converter 62 to set theinformation in the gradation converter 62. The gradation conversioncharacteristic calculated by the gradation conversion characteristiccalculator 42 is e.g. gradation conversion characteristics 410 and 420shown in FIGS. 13 and 14. The axes of abscissa and the axes of ordinatein FIGS. 13 and 14 respectively represent input in the gradationconversion, and output in the gradation conversion. The gradationconversion characteristic 410 is expressed by a curve that passes thecoordinate origin (0, 0) indicated by the reference numeral 411, and thepoint (Omax, Omax) indicated by the reference numeral 412, and alsopasses the point C (I0, I0′) indicated by the reference numeral 413, andthe point D (Ls, Ls′) indicated by the reference numeral 414. Thegradation conversion characteristic 420 is expressed by a curve thatpasses the coordinate origin (0, 0) indicated by the reference numeral411, and the point (Omax, Omax) indicated by the reference numeral 412,and also passes the point C (I0, I0′) indicated by the reference numeral413, and the point D (Ls, Ls′) indicated by the reference numeral 415.In particular, the gradation conversion characteristics 410 and 420 arecontrolled by changing the positions of the two points i.e. points C andD. In this sense, the points C and D may be defined as first and secondcontrol points, respectively.

The first control point C (I0, I0′) is a point of level at which properbrightness is attained by gamma correction. The input value I0 in thefirst control point C is a value corresponding to the luminance of e.g.a main subject e.g. the face of a person. In other words, I0 representsthe main subject luminance. The output value I0′ is an output valueapproximate to 128 gradation in the case of e.g. an 8-bit image.

The second control point D (Ls, Ls′) is a point indicating thecompression start level to be set in accordance with the gradients ofthe illumination compression characteristics 310 and 320 in FIG. 12 i.e.a gradient of a characteristic graph, specifically, the gradient of theillumination compression characteristic 310 expressed by the formula(11) i.e. a compression area or a compression characteristic, in otherwords, in accordance with the parameter representing the gradient of theillumination compression characteristic. The parameter representing thegradient of the illumination compression characteristic corresponds to aparameter to be used in the dodging process of performing compression ofthe illumination component, using the illumination compressioncharacteristic in FIG. 12, i.e. a “dodging parameter”.

A moderate gradient concerning the compression area means a larger inputarea with respect to an output area, assuming that the axis of abscissasand the axis of ordinate in FIG. 12 respectively represent input andoutput. In other words, this case means that a degree of compressioni.e. a compression ratio is large. Conversely, a steep gradientconcerning the compression area means that a degree of compression issmall. The gradient concerning the compression area is determined by twopoints i.e. the point A (Ls, Os), and another point i.e. the point E(Le, Oe) indicated by the reference numeral 313.

The second control point D (Ls, Ls′) is determined depending on theparameter representing the gradient concerning the compression area.Specifically, the second control point D (Ls, Ls′) is determined, usinginformation relating to the two points, as the parameter representingthe gradient. In this embodiment, the second control point D isdetermined, using the information relating to the two points i.e. thepoint A and the point B, in place of the point A and the point E. Inthis case, it is conceived that the point E is coincident with the pointB, or the point E is replaced by the point B. In the fourth embodiment,the information relating to the points A and B is used. Alternatively,as far as the gradient concerning the compression area can bedetermined, at least two points including the point A and another pointwhose level is larger than that of the point A may be used.Specifically, the information relating to the points A and B arerespectively Ls and Lmax, which are values of the illuminationcomponents on the axis of abscissa in FIG. 12 i.e. the aforementioneddodging parameters Ls and Lmax. The second control point D is determinedin accordance with the values of the dodging parameters Ls and Lmax.

Specifically, the input value Ls in the second control point D (Ls, Ls′)corresponds to the compression start level Ls of the illuminationcomponent L shown in FIG. 12 i.e. the dodging parameter Ls. On the otherhand, the output value Ls′ is set in accordance with the magnitude ofthe upper limit value Lmax of the illumination component L shown in FIG.12. In other words, Ls′ is dependent on Lmax, and is given by theequation: Ls′=f(Lmax). The value “f” in the equation: Ls′=f(Lmax)represents a monotonically decreasing function. “Monotonic decrease”includes any condition including linear decrease, as far as increase isnot observed. Considering the above, FIG. 13 shows a condition that Lmaxis small, and FIG. 14 shows a condition that Lmax is large. In otherwords, in the case where Lmax is small, namely, the dynamic range i.e. acompression luminance area in the dodging process is narrow (in thiscase, the gradient concerning the compression area is steep), as shownin FIG. 13, the output level of Ls′ is set high, so that a highergradation conversion output is obtained in an intermediate luminancearea or a high luminance area whose level is equal to or higher than thepoint C. On the other hand, in the case where Lmax is large, namely, thedynamic range in the dodging process is wide (in this case, the gradientconcerning the compression area is moderate), as shown in FIG. 14, theoutput level of Ls′ is set low, so that a gradation conversion output issuppressed in the intermediate luminance area or the high luminancearea. Thus, the gradation conversion characteristic is determined basedon the information relating to the compression start level Ls in thedodging process, and the luminance level Le (in this embodiment, Lmax)which is higher than the compression start level Ls.

The second control point D (Ls, Ls′), the gradation conversioncharacteristic based on the second control point D, or the conversiontable corresponds to the gradation conversion parameter. Acharacteristic curve 430 shown in FIGS. 13 and 14 represents a generallywell-known gamma characteristic. The first and second control points Cand D are controlled i.e. set in such a manner that these two controlpoints C and D do not exceed the respective corresponding levels of thegamma characteristic 430. In this embodiment, it is essential to controlthe gradation conversion characteristic based on the second controlpoint D in the aspect of setting the gradation conversion parameter inaccordance with the dodging parameter. It is, however, preferable toimplement control based on the first control point C so as to obtain amore proper image.

FIG. 15 is a flowchart showing an example of an operation of thegradation conversion process to be executed in accordance with thedodging parameter according to the fourth embodiment. First, in responseto e.g. turning on of the release switch, the image sensor 162 isoperative to acquire a sensed image to output sensed image data to themain controller 4 for storage into e.g. the image memory 181 (StepS301). Then, the dodging parameter calculator 41 of the main controller4 sends, to the dodging processor 61, the dodging parameter i.e. theinformation relating to the compression start level Ls and the upperlimit value Lmax (see FIG. 12), as well as the information relating tothe filter size Sf, for setting (Step S302).

Then, the dodging processor 61 performs a dodging process based on theset dodging parameter i.e. extracts the illumination component L fromthe original image I for dynamic range compression, and generates a newimage I′ based on the illumination component L′ and the reflectancecomponent R (Step S303). Then, the gradation conversion characteristiccalculator 42 of the main controller 4 calculates the gradationconversion characteristic (see FIGS. 13 and 14) by implementing thecomputation to obtain the second control point D (Ls, Ls′) or the likebased on the dodging parameters obtained by the dodging parametercalculator 41, creates the conversion table i.e. the LUT concerning thegradation conversion characteristic, and sends the information relatingto the created conversion table to the gradation converter 62 forsetting (Step S304). Then, the gradation converter 62 performs agradation conversion process based on the conversion table (Step S305),and the image data acquired by the gradation conversion process isstored in the image memory 181 (Step S306).

Fifth Embodiment

The fourth embodiment describes the case that the gradation conversionprocess is controlled in accordance with the dodging parameters. Thefifth embodiment describes a case that a sharpness correction process isperformed by determining a parameter (hereinafter, also called as a“sharpness parameter”) concerning sharpness correction e.g. a sharpnesscorrection amount in accordance with a dodging parameter, in otherwords, a sharpness correction process is controlled in accordance withthe dodging parameter. The block configuration of a digital camera inthe fifth embodiment is substantially the same as that shown in FIG. 10except that, as shown in FIG. 16, a main controller 4′ corresponding tothe main controller 4 in FIG. 10 is additionally provided with asharpness parameter calculator 43.

The sharpness parameter calculator 43 is adapted to calculate asharpness parameter in accordance with the area size in performing alocal computation process with respect to each of the image areas, asdescribed in the fourth embodiment, including a case that local contrastcorrection is performed as the local process. Similarly to the fourthembodiment, the area size includes the filter size and the block size tobe used in dividing an image into plural blocks. In this embodiment, thefilter size is used as the information relating to the area size. Thesharpness parameter calculator 43 calculates a sharpness parameter inaccordance with the filter size Sf of a filter to be used in extractingan illumination component, which has been calculated by the dodgingparameter calculator 41. Specifically, the sharpness parametercalculator 43 modifies a predetermined sharpness correction level, (alsodefined as a “sharpness correction amount Sh”), which has been inputtedthrough the user I/F 5, as shown in FIG. 18, in accordance with thefilter size Sf, and switches i.e. selects a filter to be used insharpness correction in accordance with the sharpness correction amountacquired by the modification.

The modification concerning the sharpness correction amount Sh inaccordance with the filter size Sf is executed by the following formula(14).Sh′=Sh+ΔSh  (14)where Sh′ is a value after modification of Sh, and ΔSh is a value i.e. amodification amount, which is given based on a relation of negativemonotonic decrease versus increase of the filter size Sf, as shown inFIG. 17. Monotonic decrease includes any condition including lineardecrease, as far as increase is not observed. In other words, as thevalue Sf increases from 0, the value ΔSh decreases from 0.

The sharpness parameter calculator 43 selects a high-pass filter forsharpness correction, which has a property that sharpness is increasedin proportion to e.g. Sh′ in the case where Sh′ is a positive value; andselects a low-pass filter for sharpness correction, which has a propertythat smoothness is increased in inverse proportion to e.g. Sh′, in thecase where Sh′ is a negative value, based on the information relating toSh, which has been obtained by implementing the formula (14). In otherwords, the sharpness parameter calculator 43 changes a frequencycharacteristic for edge extraction to be used in a sharpness correctionprocess by the sharpness corrector 63 in accordance with the sharpnesscorrection amount Sh, which is determined based on the filter size Sfi.e. the area size based on which the local computation process isperformed.

In the case where the value Sh′ is 0 in the formula (14), the filterprocess for the sharpness correction is not executed. In other words, nofilter is selected. The information relating to the filters such as thehigh-pass filter and the low-pass filter may be stored in advance in thesharpness parameter calculator 43 or a like device. The sharpnessparameter calculator 43 sends the information relating to the selectedfilter to the sharpness corrector 63 for setting the information in thesharpness corrector 63. The sharpness corrector 63 executes thesharpness correction process based on the information relating to theset filter i.e. one of the high-pass filter and the low-pass filter. Thesharpness correction amount Sh′ and the filter selected based on thesharpness correction amount Sh′ correspond to the sharpness parameter.

FIG. 19 is a flowchart showing an example of an operation of thesharpness correction process to be executed in accordance with thedodging parameter in the fifth embodiment. First, the informationrelating to the sharpness correction amount Sh, which has been inputtedby the user through the user I/F 5 is set in the sharpness parametercalculator 43 (Step S401). Then, in response to e.g. turning on of therelease switch, the image sensor 162 is operative to acquire a sensedimage to output sensed image data to the main controller 4′ for storageinto e.g. the image memory 181 (Step S402). Then, the dodging parametercalculator 41 of the main controller 4′ sends, to the dodging processor61, the dodging parameters, i.e. the information relating to thecompression start level Ls and the upper limit value Lmax (see FIG. 3),as well as the information relating to the filter size Sf, for setting(Step S403). Then, the dodging processor 61 performs the dodgingprocess, based on the set dodging parameters i.e. extracts theillumination component L from the original image I for dynamic rangecompression, and generates a new image I′ based on the illuminationcomponent L′ and the reflectance component R (Step S404).

Next, the gradation conversion characteristic calculator 42 of the maincontroller 4′ calculates the gradation conversion characteristic (seeFIGS. 4 and 5) by computation to obtain the second control point D (Ls,Ls′) or the like, based on the dodging parameters obtained by thedodging parameter calculator 41, and creates the conversion table i.e.LUT concerning the gradation conversion characteristic. Then, theinformation relating to the created conversion table is sent to thegradation converter 62 for setting (Step S405). Then, the gradationconverter 62 performs the gradation conversion process based on theconversion table (Step S406). Then, the sharpness parameter calculator43 of the main controller 4′ modifies the sharpness correction amount Shinputted in Step S301 in accordance with the filter size Sf obtained bythe dodging parameter calculator 41 (see the formula (14)).

Then, the filter to be used in sharpness correction is selected inaccordance with the sharpness correction amount Sh′ after themodification, and the selected filter is set in the sharpness corrector63 (Step S407). Then, the sharpness corrector 63 performs the sharpnesscorrection process based on the set filter (Step S408). The image dataobtained by the sharpness correction process is stored into the imagememory 181 (Step S409). The gradation conversion process in Steps S404and S406 may not be necessarily executed in accordance with the dodgingparameter. An ordinary well-known gradation conversion process usinggamma may be executed, or a flow without the gradation conversionprocess, in which the routine proceeds directly from Step S404 to StepS407, may be executed. The order of implementing Steps S405, S406, S407,and S408 is not limited to the foregoing, but may be changed, forinstance, in such a manner that Steps S405 and S406 are executed afterSteps S407 and S408 are executed.

As mentioned above, the digital camera 1 or 1 a recited in the firstthrough the fifth embodiments, as an image sensing apparatus, includes:the image sensor 162, or the image sensor 162 and the imaging circuit160, for sensing an image of a subject i.e. a subject light image, as animage sensing section; the photometry module 122, or the photometrymodule 122 and the main controller 150, or the main controller 4, fordetecting a luminance of the subject, as a detector; the dynamic rangecompressor 504, or the dodging processor 61, for compressing a dynamicrange of the subject image, as a compressor; and the main controller150, or the main controller 4, for controlling a compressioncharacteristic to be used in compressing the dynamic range based on adetection result of the detector, as a controller. This enables tocontrol the compression characteristic to be used in compressing thedynamic range based on the subject luminance information detected by thedetector. This is advantageous in obtaining a proper image withoutgeneration of an unnatural image, in which the contrast of the subjectimage in a high luminance area may be unduly lowered, or a highfrequency component i.e. an edge component may be emphasized, withoutimpairing the useful information included in the original image. Inother words, a proper image i.e. a high-quality image can be obtaineddespite the dynamic range compression. The parameter for e.g. thedynamic range compression i.e. for controlling the compressioncharacteristic is calculated based on the subject luminance informationobtained in sensing the original image. This allows for high speedacquisition of the parameter to be used in the dynamic rangecompression.

The detector detects the luminance of the subject in a plurality ofareas of the subject image e.g. the photometric sections A0, and A1through A13. The controller controls the compression characteristic e.g.the compression characteristic curves CC1 and CC2 based on a maximalvalue of the detected subject luminances i.e. the maximal luminanceBvmax, and a minimal value of the detected subject luminances i.e. theminimal luminance Bvmin in the plurality of areas e.g. the photometricsections A1 through A13 i.e. based on the multi-area photometric data.This enables to facilitate control of the compression characteristic tobe used in the dynamic range compression, based on the detected subjectluminance information i.e. the maximal and minimal values of the subjectluminance.

In the case where there exist a first difference value D1(=Bvmax1−Bvmin1) and a second difference value D2 (=Bvmax2−Bvmin2)smaller than the first difference value, as a difference value betweenthe maximal value and the minimal value, the controller controls thecompression characteristic in such a manner that a compression ratioe.g. c2 set for the second difference value is larger than a compressionratio e.g. c1 set for the first difference value. This facilitatescontrol of the compression characteristic to be used in the dynamicrange compression, using the maximal and minimal values of the subjectluminance.

The detector detects the luminance in at least two areas concerning thesubject image e.g. a main subject area and a background area. Thecontroller controls the compression characteristic based on adiscrimination result as to whether one of the detected subjectluminances i.e. the main subject luminance Bvmain and the backgroundluminance Bvsub in the two areas is larger or smaller than the other oneof the detected subject luminances. This facilitates control of thecompression characteristic to be used in the dynamic range compression,based on the detected subject luminance information, in other words, atleast two luminance values i.e. the main subject luminance and thebackground luminance concerning the subject luminance.

One of the two areas is the main subject area corresponding to a centralarea of the subject image e.g. the central area A7, and the other one ofthe two areas is the background area corresponding to an area other thanthe central area of the subject image e.g. the area other than thecentral area A7. The controller controls the compression characteristicin such a manner that a first compression ratio c1, a second compressionratio c3, and a third compression ratio c2 satisfy the relationship:c1<c2<c3, wherein the first compression ratio c1 is used in the casewhere a first different value (Bvmain−Bvsub) obtained by subtracting thebackground luminance in the background area from the main subjectluminance in the main subject area is equal to or larger than zero, thesecond compression ratio c3 is used in the case where a second differentvalue (Bvmain−Bvsub) obtained by subtracting the background luminancefrom the main subject luminance is equal to or smaller than apredetermined negative threshold value i.e. the backlit discriminationvalue TH, and the third compression ratio c2 is used in the case where athird difference value (Bvmain−Bvsub) obtained by subtracting thebackground luminance from the main subject luminance is smaller thanzero and is larger than the threshold value. This facilitates control ofthe compression characteristic to be used in the dynamic rangecompression, using the luminance values in the at least two areas.

The image sensing section includes an image sensor i.e. the image sensor162, which has a photoelectric conversion characteristic with a linearcharacteristic where an electric signal is linearly convertedcommensurate with an incident light amount for output, and a logarithmiccharacteristic where the electric signal is logarithmically convertedcommensurate with the incident light amount for output, and which isoperative to obtain a linear-logarithmic image by a sensing operationthereof. With use of the image sensor, a wide dynamic range image can beeasily obtained, which facilitates acquisition of a proper image withits compression characteristic to be used in the dynamic rangecompression being controlled based on the subject luminance information,using the wide dynamic range image.

The digital camera 1 may further comprise: a characteristic converter,i.e. the main controller 150 or the image processor 165, for convertingthe linear-logarithmic image into a linear image having the linearcharacteristic over the entirety of the photoelectric conversioncharacteristic; an image creator, i.e. the main controller 150 or theimage processor 165, for creating, based on the linear image, aplurality of images having different contrasts in accordance with thedetected luminance in each of a plurality of areas concerning thesubject image; an image divider, i.e. the main controller 150 or theimage processor 165, for dividing each of the plurality of images into aplurality of block images; and an image synthesizer, i.e. the maincontroller 150 or the image processor 165, for selecting the block imagehaving a maximal contrast from the block images at positionscorresponding to each other among the plurality of images with respectto each of the block images for synthesis of the selected block images.This enables to select a block image with a sufficiently high contrastwith respect to each of the block images, thereby enabling to obtaininga proper image in which the high contrast block images are synthesized.A natural image with no or less incongruity between the block images canbe obtained by executing the aforementioned blend process insynthesizing the block images, which enables to obtain a high-qualityimage.

Another aspect of the invention is directed to an image sensing methodrecited in the first through the fifth embodiments. The image sensingmethod comprises an image sensing step of sensing an image of a subject;a detecting step of detecting a luminance of the subject in a pluralityof areas concerning the subject image; a calculating step of calculatinga compression characteristic to be used in compressing a dynamic rangeof the subject image based on a detection result obtained in thedetecting step; and a compressing step of compressing the dynamic rangeof the subject image using the compression characteristic. This enablesto control the compression characteristic to be used in the dynamicrange compression in the compressing step, based on the subjectluminance information obtained in the detecting step. This isadvantageous in obtaining a proper image without generation of anunnatural image, in which the contrast of the subject image in a highluminance area may be unduly lowered, or a high frequency component i.e.an edge portion may be emphasized, without impairing the usefulinformation included in the original image. In other words, a properimage i.e. a high-quality image can be obtained despite the dynamicrange compression.

The image sensing apparatus recited in the fourth embodiment i.e. thedigital camera 1 a is constructed in such a manner that the image inputsection 2 or the image sensor 162, as the image sensing section, obtainsa subject image; the dodging processor 61, as a processor, performs adodging process concerning the dynamic range compression with respect tothe subject image; the dodging parameter calculator 41, as a parametersetter, sets the dodging parameter, as a first parameter, concerning thedodging process, to control the compression characteristic; thegradation converter 62, as a converter, performs a gradation conversionprocess with respect to the subject image; and the gradation conversioncharacteristic calculator 42, as a gradation characteristic setter, setsa gradation conversion characteristic concerning the gradationconversion process. The gradation conversion characteristic calculator42 determines the gradation conversion characteristic based on thedodging parameter set by the dodging parameter calculator 41, and thegradation converter 62 performs the gradation conversion process basedon the gradation conversion characteristic determined by the gradationconversion characteristic calculator 42.

In the above arrangement, the gradation conversion characteristic isdetermined based on the dodging parameter, and the gradation conversionprocess is performed based on the determined gradation conversioncharacteristic. This enables to obtain a proper gradationcharacteristic, considering a characteristic inherent to the dodgingprocess without incurring contrast lowering in the high luminance areaas a result of the gradation conversion. In other words, despite thedodging process i.e. the dynamic range compression, a proper image i.e.a high-quality image having a proper gradation characteristic,considering the dodging characteristic, can be obtained withoutincurring contrast lowering in the high luminance area as a result ofthe gradation conversion.

The dodging processor 61 performs the dodging process by compressing theillumination component L extracted from the original image I, and bygenerating the new image I′ based on the compressed illuminationcomponent L′ and the reflectance component R extracted from the originalimage, and the gradation conversion characteristic calculator 42determines the gradation conversion characteristic 410 (420) based onthe parameter representing a degree of compression in the luminancerange where the illumination component L is to be compressed. Thisenables to determine the gradation conversion characteristic 410 (420),while properly considering the dodging characteristic.

The gradation conversion characteristic calculator 42 determines thegradation conversion characteristic 410 (420) based on information, asthe dodging parameter, relating to a first luminance level correspondingto the predetermined compression start level Ls in the luminance rangewhere the illumination component is to be compressed, and a secondluminance level higher than the compression start level Ls. Thisfacilitates determination of the gradation characteristic 410 (420),while properly considering the dodging characteristic, using theinformation relating to the first luminance level, which is set to alevel value obtainable of proper brightness or proper contrastconcerning e.g. the main subject, and the second luminance level higherthan the first luminance level.

The first luminance level may be a luminance level equal to or largerthan a main subject luminance corresponding to a predetermined luminancevalue of the main subject, i.e. the input value I0 in the gradationconversion shown in FIGS. 13 and 14. In the case where the firstluminance level is defined as an input value Ls in the gradationconversion, the gradation conversion characteristic calculator 42determines the gradation conversion characteristic 410 (420) in such amanner that an output value Ls′ in the gradation conversion with respectto the gradation conversion input value Ls is set to a small value ifthe degree of compression of the illumination component L is large, andthat the gradation conversion output value Ls′ with respect to thegradation conversion input value Ls is set to a large value if thedegree of compression of the illumination component L is small. Thus,proper gradation conversion considering the dodging characteristic canbe executed in such a manner that: in the case where the degree ofcompression of the illumination component L is large, in other words,the luminance area for compression i.e. the dynamic range in the dodgingprocess is wide, the gradation conversion output in the intermediateluminance area or the high luminance area is suppressed by setting thegradation conversion output to a low level (see the example shown inFIG. 14); and in the case where the degree of compression of theillumination component L is small, in other words, the luminance areafor compression in the dodging process is narrow, a higher gradationconversion output is obtained in the intermediate luminance area or thehigh luminance area by setting the gradation conversion output to a highlevel (see the example shown in FIG. 13).

The gradation conversion characteristic calculator 42 may determine thegradation conversion characteristic 410 (420) capable of converting thegradation conversion input value I0 concerning the main subjectluminance into a proper output value I0′ in the gradation conversion,which is an output level obtainable of intended brightness concerningthe main subject. This enables to prevent likelihood that the mainsubject image may be dark as a result of the gradation conversion,thereby enabling to perform more proper gradation conversion.

The image sensing apparatus recited in the fifth embodiment i.e. thedigital camera 1 a is constructed in such a manner that the image inputsection 2 or the image sensor 162, as the image sensing section, obtainsthe subject image; the dodging processor 61, as the processor, performsthe dodging process concerning the dynamic range compression withrespect to the subject image; the dodging parameter calculator 41, as aparameter setter, sets the dodging parameter, as a first parameterconcerning the dodging process, to control the compressioncharacteristic; the sharpness corrector 63 performs a sharpnesscorrection process with respect to the subject image; and the sharpnessparameter calculator 43, as a correction amount setter, sets a sharpnesscorrection amount Sh concerning the sharpness correction process. Thesharpness parameter calculator 43 determines the sharpness correctionamount Sh based on the dodging parameter set by the dodging parametersetter 41, and the sharpness corrector 63 performs the sharpnesscorrection process based on the sharpness correction amount Shdetermined by the sharpness parameter calculator 43.

In the above arrangement, the sharpness correction amount Sh isdetermined based on the dodging parameter, and the sharpness correctionprocess is performed based on the set sharpness correction amount Sh.This enables to obtain a proper sharpness characteristic, consideringthe dodging characteristic, without generation of an unnatural image, inwhich a high frequency component may be emphasized as a result of thedodging process. In other words, despite the dodging process i.e. thedynamic range compression, a proper image i.e. a high-quality imagehaving a proper sharpness characteristic, considering the dodgingcharacteristic, can be obtained without generation of an unnaturalimage, in which the high frequency component may be emphasized as aresult of the dodging process.

The dodging process may be realized by performing a computation processlocally with respect to a plurality of image areas obtained by dividingthe subject image i.e. the original image I, and the dodging parametermay be a size i.e. the area size of each of the image areas to which thelocal computation process is to be performed. In this arrangement, sincethe area size to be used in the local computation process concerning thesubject image is handled as the dodging parameter, various parameterssuch as the filter size to be used in performing a filter process withrespect to the subject image, or the block size to be used in dividingthe original image into a plurality of blocks, can be used as thedodging parameter. This enables to obtain a proper sharpnesscharacteristic in accordance with the dodging parameter, consideringthese parameters.

The dodging parameter may be the filter size Sf of a filter i.e. theedge preserving filter for extracting the illumination component L fromthe original image. The sharpness parameter calculator 43 determines thesharpness correction amount Sh based on the filter size Sf (see theformula (5)). In this arrangement, since the sharpness correction amountis determined based on the filter size Sf, the sharpness correctionamount can be easily determined, considering the dodging parameter, inperforming the sharpness correction process in accordance with thedodging characteristic.

The dodging process may be realized by performing a contrast correctionprocess locally with respect to a plurality of image areas obtained bydividing the subject image i.e. the original image I, and the dodgingparameter may be a size of each of the image areas to which the localcontrast correction process is to be performed. In this arrangement,since the area size to be used in the local contrast correction processwith respect to the subject image is handled as the dodging parameter, aproper sharpness characteristic can be obtained in accordance with thedodging parameter, considering the local contrast correction process.

The sharpness parameter calculator 43 changes a frequency characteristicfor edge extraction to be used by the sharpness corrector 63 inaccordance with the sharpness correction amount determined based on thesize of the each of the image areas to which the local computationprocess is to be performed. In this arrangement, the frequencycharacteristic for edge extraction to be used by the sharpness corrector63 is changed in accordance with the sharpness correction amountdetermined based on the area size, in other words, the frequencycharacteristic for edge extraction in accordance with the sharpnesscorrection amount is used. This enables to perform the sharpnesscorrection based on the edge extraction more properly, considering thedodging characteristic.

The sharpness parameter calculator 43 may change an edge emphasis amountto be used by the sharpness corrector 63 in accordance with thesharpness correction amount determined based on the size of the each ofthe image areas to which the local computation process is to beperformed. In this arrangement, the edge emphasis amount to be used bythe sharpness corrector 63 is changed in accordance with the sharpnesscorrection amount determined based on the area size, in other words, theedge emphasis amount in accordance with the sharpness correction amountis used. This enables to perform the sharpness correction based on theedge emphasis more properly, considering the dodging characteristic.

The image sensing section may include the image sensor 162 i.e. thelinear-logarithmic sensor having the photoelectric conversioncharacteristic with the linear characteristic where an electric signalis linearly converted commensurate with an incident light amount foroutput, and the logarithmic characteristic where the electric signal islogarithmically converted commensurate with the incident light amountfor output. In this arrangement, a wide dynamic range image can beeasily obtained by a sensing operation of the image sensor 162. This isadvantageous in easily obtaining an image, with its brightness orcontrast in the main subject i.e. the low luminance area being secured,and its brightness or contrast in the high luminance area being properlyadjusted, based on the wide dynamic range image, in performing thegradation conversion process based on the dodging parameter.

The arrangement recited in the fourth embodiment, in which the gradationconversion characteristic is determined based on the dodging parameter,and the gradation conversion process is performed based on thedetermined gradation conversion characteristic, and the arrangementrecited in the fifth embodiment, in which the sharpness correctionamount is determined based on the dodging parameter, and the sharpnesscorrection process is performed based on the determined sharpnesscorrection amount may be realized by the below-mentioned imageprocessing method or image processing program.

Specifically, the image processing method includes: a dodging processperforming step of performing a dodging process with respect to imagedata; a dodging parameter setting step of setting a dodging parameterconcerning the dodging process; a gradation conversion characteristicdetermining step of determining a gradation conversion characteristicbased on the set dodging parameter; and a gradation conversion processperforming step of performing a gradation conversion process based onthe determined gradation conversion characteristic.

The image processing program causes a computer to function as a dodgingparameter setter for setting a dodging parameter concerning a dodgingprocess with respect to image data, and a gradation conversioncharacteristic determiner for determining a gradation conversioncharacteristic based on the dodging parameter set by the dodgingparameter setter.

With use of the image processing method or the image processing programas mentioned above, the gradation conversion characteristic isdetermined based on the dodging parameter, and the gradation conversionprocess is performed based on the determined gradation conversioncharacteristic. This enables to obtain a proper gradationcharacteristic, considering the dodging characteristic, withoutincurring contrast lowering in the high luminance area as a result ofthe gradation conversion.

The image processing method may include: a dodging process performingstep of performing a dodging process with respect to image data; adodging parameter setting step of setting a dodging parameter concerningthe dodging process; a sharpness correction amount determining step ofdetermining a sharpness correction amount based on the set dodgingparameter; and a sharpness correction process performing step ofperforming a sharpness correction process based on the determinedsharpness correction amount.

The image processing program may cause a computer to function as adodging parameter setter for setting a dodging parameter concerning adodging process with respect to image data, and a sharpness correctionamount determiner for determining a sharpness correction amount based onthe dodging parameter set by the dodging parameter setter.

With use of the image processing method or the image processing programas mentioned above, the sharpness correction amount is determined basedon the dodging parameter, and the sharpness correction process isperformed based on the determined sharpness correction amount. Thisenables to obtain a proper sharpness characteristic, considering thedodging characteristic, without generation of an unnatural image, inwhich a high frequency component may be emphasized as a result of thedodging process.

The following modifications and/or alterations may be applied to theinvention.

(A) The dynamic range compression process and the compressioncharacteristic control in the first through the fifth embodiment; andthe gradation conversion process and the sharpness correction process inaccordance with the dodging parameter as well as the dynamic rangecompression process and the compression characteristic control in thefourth and the fifth embodiment are executed by the parts provided inthe image sensing apparatus i.e. the digital camera 1 or 1 a.Alternatively, these processes may be executed by a predeterminedinformation processor provided outside of the image sensing apparatus.Specifically, a predetermine host processor e.g. a PC (PersonalComputer), or a computer-built-in device such as a PDA (Personal DigitalAssistant) may execute the processes. The host processor isnetwork-connected with the digital camera 1 or 1 a by direct connectioni.e. wired connection using a USB or a like device, or a wireless LAN,or a like device; or is provided with a user interface which isconfigured to transmit information, using a storage medium such as amemory card, or a like medium. The use of the information processor i.e.an external device externally connected to the digital camera 1 or 1 aenables to perform a fine compression process of an illuminationcomponent, or a gradation conversion process or a sharpness correctionprocess in accordance with a dodging parameter.

In the modification (A), the image sensing apparatus and the informationprocessor may constitute an image sensing system. Specifically, theimage sensing system comprises: an image sensing apparatus i.e. thedigital camera 1 or 1 a for sensing an image of a subject; and aninformation processor e.g. the PC for performing a predeterminedcomputation process, wherein the image sensing apparatus and/or theinformation processor includes a detector for detecting a luminance ofthe subject, and the information processor includes a compressor forcompressing a dynamic range of the subject image, and a controller forcontrolling a compression characteristic to be used in compressing thedynamic range based on a detection result of the detector. In the imagesensing system having the above arrangement, the compressioncharacteristic to be used in compressing the dynamic range can becontrolled based on the subject luminance information detected by thedetector. This enables to obtain a proper image without generation of anunnatural image, in which the contrast of the subject image in a highluminance area may be unduly lowered, or a high frequency component i.e.an edge component may be emphasized, without impairing the usefulinformation included in the original image. In other words, a properimage i.e. a high-quality image can be obtained despite the dynamicrange compression.

(B) In the fifth embodiment, the sharpness parameter is calculated bycorrecting the sharpness correction amount Sh in accordance with thefilter size Sf of the filter for extracting an illumination componenti.e. the area size. Alternatively, an outline emphasis amount i.e. anedge emphasis amount with respect to an image obtained by a coringprocess may be changed in accordance with the filter size Sf i.e. thearea size. Specifically, the sharpness corrector 63 may include, asshown in FIG. 20 for instance, an outline emphasis processor 64. Theoutline emphasis processor 64 has a filter processor 641, an adder 642,a coring processor 643, an outline emphasis amount setter 644, amultiplier 645, and an adder 646 so that the outline emphasis amountsetter 644 sets a gain i.e. an outline emphasis amount in accordancewith the filter size Sf, and the multiplier 645 multiplies the gain by acoring result outputted from the coring processor 643 i.e. an edge image654 to be described later for outline emphasis. In this modification,the outline emphasis amount is set to a value proportional to the filtersize Sf.

The filter processor 641 performs a two-dimensional LPF (low-passfilter) process with respect to e.g. an input image 651 i.e. an originalimage, using a filter of symmetry type having M×N e.g. 5×5 in size, andoutputs an LPF image 652 corresponding to a low frequency component. Theadder 642 computes a difference between the input image 651 and the LPFimage 652, and extracts a high frequency component i.e. a differenceimage 653 having an edge component and a noise component. The coringprocessor 643 performs a coring process i.e. a noise component removalprocess by removing a coring factor from the difference image 653 tooutput an edge image 654. The coring factor is a factor to be used inremoving the noise component from the image signal for extraction of theedge component. The outline emphasis amount setter 644 sets an outlineemphasis amount in accordance with the filter size Sf calculated by thedodging parameter calculator 41. The multiplier 645 multiplies the edgeimage 654 by the outline emphasis amount set by the outline emphasisamount setter 644, and outputs an edge emphasis image 655. In thismodification, the outline emphasis amount is set to about 6. The adder646 multiplies the edge emphasis image 655 by the LPF image 652outputted from the filter processor 641 for synthesis, and outputs anoutput image 656. Thus, the outline emphasis processor 64 is operativeto generate the output image 656, in which the noise component isremoved from the input image 651, and the edge component is emphasized.

Although the invention has been appropriately and fully described by wayof examples with reference to the accompanying drawings, it is to beunderstood that various changes and/or modifications will be apparent tothose skilled in the art. Therefore, unless otherwise such changesand/or modifications depart from the scope of the present inventionhereinafter defined, they should be construed as being included therein.

1. An image sensing apparatus comprising: an image sensing section forsensing an image of a subject; a detector for detecting a luminance ofthe subject; a compressor for compressing a dynamic range of the subjectimage; and a controller for controlling a compression characteristic tobe used in compressing the dynamic range based on a detection result ofthe detector; a processor for performing a dodging process concerningthe dynamic range compression with respect to the subject image; aparameter setter for setting a first parameter concerning the dodgingprocess to control the compression characteristic; a converter forperforming a gradation conversion process with respect to the subjectimage; and a gradation characteristic setter for setting a gradationconversion characteristic concerning the gradation conversion process,wherein the gradation characteristic setter determines the gradationconversion characteristic based on the first parameter, and theconverter performs the gradation conversion process based on thegradation conversion characteristic.
 2. The image sensing apparatusaccording to claim 1, wherein the processor performs the dodging processby compressing an illumination component extracted from the subjectimage, and generating a new image based on the compressed illuminationcomponent and a reflectance component extracted from the subject image,and the gradation characteristic setter determines the gradationconversion characteristic based on the first parameter representing adegree of compression in a luminance range where the illuminationcomponent is to be compressed.
 3. The image sensing apparatus accordingto claim 2, wherein the gradation conversion setter determines thegradation conversion characteristic based on information, as the firstparameter, relating to a first luminance level corresponding to apredetermined compression start level in the luminance range where theillumination component is to be compressed, and a second luminance levelhigher than the compression start level.
 4. The image sensing apparatusaccording to claim 3, wherein the first luminance level is a luminancelevel equal to or larger than a main subject luminance corresponding toa predetermined luminance value of a main subject image, and in the casewhere the first luminance level is defined as an input value for thegradation conversion process, the gradation characteristic setterdetermines the gradation conversion characteristic in such a manner thatan output value for the gradation conversion process with respect to thegradation conversion input value is set to a small value if the degreeof compression of the illumination component is large, and that thegradation conversion output value with respect to the gradationconversion input value is set to a large value if the degree ofcompression of the illumination component is small.
 5. The image sensingapparatus according to claim 1, wherein in the case where a properoutput value for the gradation conversion process concerning a mainsubject image is set to I0′, the gradation characteristic setterdetermines, when an input value for the gradation conversion processconcerning a main subject luminance is I0, the gradation conversioncharacteristic that enables to convert I0 to I0′.
 6. An image sensingapparatus comprising: an image sensing section for sensing an image of asubject; a detector for detecting a luminance of the subject; acompressor for compressing a dynamic range of the subject image; and acontroller for controlling a compression characteristic to be used incompressing the dynamic range based on a detection result of thedetector; a processor for performing a dodging process concerning thedynamic range compression with respect to the subject image; a parametersetter for setting a first parameter concerning the dodging process tocontrol the compression characteristic; a sharpness corrector forperforming a sharpness correction process with respect to the subjectimage; and a correction amount setter for setting a sharpness correctionamount concerning the sharpness correction process, wherein thecorrection amount setter determines the sharpness correction amountbased on the first parameter, and the sharpness corrector performs thesharpness correction process based on the sharpness correction amount.7. The image sensing apparatus according to claim 6, wherein the dodgingprocess is realized by performing a computation process locally withrespect to a plurality of image areas obtained by dividing the subjectimage, and the first parameter is a size of each of the image areas towhich the local computation process is to be performed.
 8. The imagesensing apparatus according to claim 6, wherein the processor performsthe dodging process by compressing an illumination component extractedfrom the subject image, and generating a new image based on thecompressed illumination component and a reflectance component extractedfrom the subject image, and the first parameter is a size of a filterfor extracting the illumination component from the subject image, andthe correction amount setter determines the sharpness correction amountbased on the size of the filter.
 9. The image sensing apparatusaccording to claim 6, wherein the dodging process is realized byperforming a contrast correction process locally with respect to aplurality of image areas obtained by dividing the subject image, and thefirst parameter is a size of each of the image areas to which the localcontrast correction process is to be performed.
 10. The image sensingapparatus according to claim 7, wherein the correction amount setterchanges a frequency characteristic for edge extraction to be used by thesharpness corrector in accordance with the sharpness correction amountdetermined based on the size of the each of the image areas to which thelocal computation process is to be performed.
 11. The image sensingapparatus according to claim 7, wherein the correction amount setterchanges an edge emphasis amount to be used by the sharpness corrector inaccordance with the sharpness correction amount determined based on thesize of the each of the image areas to which the local computationprocess is to be performed.